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Method of rendering graphical objects 

FIELD OF THE INVENTION 

The present invention relates to a method of and a device for mapping an input image 

5 onto an output image. 

This invention may be used in digital apparatuses adapted to render graphical objects 
such as, for example, video decoders, 3D graphic accelerators, video game consoles, personal 
digital assistants or mobile phones. 

10 BACKGROUND OF THE INVENTION 

Texture mapping is a process for mapping an input image onto a surface of a 
graphical object to enhance the visual realism of a generated output image including said 
graphical object. Intricate detail at the surface of the graphical object is very difficult to 
model using polygons or other geometric primitives, and doing so can greatly increase the 

1 5 computational cost of said object. Texture mapping is a more efficient way to represent fine 
detail on the surface of the graphical object. In a texture mapping operation, a texture data 
item of the input image is mapped onto the surface of the graphical object as said object is 
rendered to create the output image. 

In conventional digital images, the input and output images are sampled at discrete 

20 points, usually on a grid of points with integer coordinates. The input image has its own 

coordinate space (u,v). Individual elements of the input image are referred to as "texels." Said 
texels are located at integer coordinates in the input coordinate system (u,v). Similarly, the 
output image has its own coordinate space (x,y). Individual elements of the output image are 
referred to as "pixels." Said pixels are located at integer coordinates in the output coordinate 

25 system (x,y). 

The process of texture mapping conventionally includes filtering texels from the input 
image so as to compute an intensity value for a pixel in the output image. Conventionally, the 
input image is linked to the output image via an inverse affine transform T 1 that can be 
expressed by the following equations: 



30 



9u , 3u , 
u=u cef +— -dx+— dy 

9x dy (1) 

dv , dv , 
v=Vref+ _ dx+ _- dy 
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where (u,v) axe the coordinates of an input current point, Ow,v re f) are the coordinates 
3u 3u 3 3 

of an input reference point, — are affine coefficients, and dx - x - x ref 

dx dy dx dy ? 

dy^y-yref. 

Similarly, the output image is linked to the input image via a direct affine transform T 
that can be expressed by the following equations: 

3x , dx. - 

x = x ref +— du+— dv 
du dv 

3y^ 9y, < 2) 
v = y ref +^du+^dv 
ou dv 

where (x,y) are the coordinates of an output current point, (* e f,yref) are the coordinates 

3tx 3 3x 3 

of an output reference point, — and~^ are affine coefficients, and du = u - iw 

du du dv dv ' 

dv= V— Vref. 

Equations (1) are called the backward mapping equations (i.e. the input coordinates 
are expressed as a function of the output coordinates) and equations (2) are called the forward 
mapping equations (i.e. the output coordinates are expressed as a function of the input 
coordinates). 

As explained above, the output image is made of a plurality of triangles defined by the 
positions of their vertices in both output and input coordinate systems. Said positions define a 
unique affine transform between a triangle in the input image and a triangle in the output 
image. To generate the output image, each output triangle is scartconverted to calculate the 
intensity value of each pixel of the triangle on the basis of intensity values of texels. 

The texels to be filtered for a given pixel are determined thanks to the inverse affine 
transform T 1 . The result of the inverse affine transform of the given pixel is a transformed 
point in the input image. Texels neighboring said transformed point in the input image are 
then sampled, weighted and summed to compute the intensity value of the given pixel. This 
process is repeated for all the pixels in the output image. 

In order to make the scan-conversion relatively simple with only incremental 

computations, an expensive computational setup is required. This setup computes the affine 

™. . 9u 3u 9v 1 3v . 
coefficients — — and— and output triangle edge slopes a 0 i, ao2 and aj 2 . If the 

coordinates of the vertices of the input triangle are (uO,vO), (ul,vl) and (u2,v2) and if the 
coordinates of the vertices of the output triangle are (xO,yO), (xl,yl) and (x2,y2), said 
coefficients and slopes are determined from the equations (3) and (4), respectively: 
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(3) 



(4) 



A hardware system being able to compute these coefficients requires large arithmetic 
operators. This increases the silicon area needed for the implementation of said operators and 
5 thereafter the price of the hardware system. 

SUMMARY OF THE INVENTION 

It is an object of the invention to propose a method of and a device for mapping an 
input image onto an output image, which is less computational expensive than the one of the 
10 prior art. 

To this end, the mapping method in accordance with the invention is characterized in 
that it comprises the steps of: 

detemiining an inverse affine transform for transforming an intermediate rectangle 
triangle into an input triangle; 
15 - determining a direct affine transform for transforming the intermediate rectangle 
triangle into an output triangle; 

applying the inverse affine transform to intermediate points of the intermediate 
rectangle triangle so as to determine intermediate intensity values corresponding to said 
intermediate points on the basis of input intensity values of texels; and 
20 - applying the direct affine transform to the intermediate points so as to determine 
output intensity values of pixels on the basis of the intermediate intensity values. 

As it will be explained in more detail hereinafter, the introduction of an intermediate 
space between the input imags and the output image makes possible to determine easily the 



PHFR040005EPp 



inverse affine transform and the direct affine transform, contrary to the prior art where the 
detemiination of the inverse or direct affine transform is more complex. As a consequence, 
the mapping method in accordance with the invention allows silicon area to be saved and less 
power to be consumed. 

5 The present invention also relates to a device implementing such a mapping method 

and to a portable apparatus comprising said device. 

Said invention finally relates to a computer program product comprising program 
instructions for implementing said mapping method. 

These and other aspects of the invention will be apparent from and will be elucidated 
10 with reference to the embodiments described hereinafter. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will now be described in more detail, by way of example, with 
reference to the accompanying drawings, wherein: 
15 - Figure 1 shows an image comprising triangle meshes; 

Figure 2 illustrates a first embodiment of the mapping method in accordance with the 
invention; 

Figure 3 shows a filtering step based on finite impulse response FIR filters; 
Figure 4 illustrates another embodiment of the mapping method in accordance with 
20 the invention; 

Figure 5 illustrates a particular case of transformation between an intermediate space 
and an output image; 

Figure 6 shows another embodiment of the mapping method in accordance with the 
invention to solve the particular case of Figure 5; and 
25 - Figure 7 shows a block diagram of a rendering device in accordance with the 
invention. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention relates to a method of mapping texture data> i.e. luminance or 
30 chrominance data, of an input image split into input triangles onto an output image also split 
into corresponding output triangles. Input and output triangle meshes can be made of any 
possible triangle as shown in Figure 1. 

As neither output mesh nor input mesh can be regular for a backward mapping 
corresponding to equations (1) or a forward mapping corresponding to equations (2), 
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respectively, an additional scan-conversion space (s,t), hereinafter referred to as intermediate 
space, is introduced. The current positions (x,y) and (u,v) in the output image and in the input 
image, respectively, are computed on the basis of a current point in said intermediate space. 
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Figure 2 illustrates a first embodiment of the mapping method in accordance with the 
invention. The intermediate space comprises an intermediate rectangle triangle TO having 
three vertices (sO^O^^O), (sl,tl)=(b,0) and (s2,t2)=(0,a). An input triangle Tl having 
vertices (u0,v0), (ul,vl) and (u2,v2) is linked to the intermediate rectangle triangle TO via an 
inverse affine transform BT that can be expressed by the backward mapping equations (1). 
Thanks to the use of the intermediate rectangle triangle, the calculation of the affine 

3v 

— is simplified as expressed by equations (5): 
dt 
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(5) 



Similarly, an output triangle T2 having vertices (x0,y0), (xl,yl) and (x2,y2) is linked 
to the intermediate rectangle triangle TO via an inverse affine transform FT that can be 
expressed by the forward mapping equations (2). Thanks to the use of the rectangle triangle, 

the calculation of the affine coefficients — ,— ,— and * s a ^ so simplified as expressed by 

3s dt as dt 

equations (6): 
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(6) 



As a consequence two sets of affine coefficients are computed: 

one for the backward mapping between the intermediate space and the input image, 



and 
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one for the forward mapping between the intermediate space and the output image. 
These sets of coefficients are computed in a relatively computation- free setup, saving silicon 
area and consuming less power. 

5 Once the two sets of coefficients have been computed, the mapping method in 

accordance with the invention is adapted to determine, for an intermediate point pO(n) of the 
intermediate rectangle triangle TO, n being an integer, an input transformed point pl(n) in the 
input triangle Tl using the inverse affine transform BT. The input transformed point pl(n) 
belonging to the input triangle is in general not located on a grid of texels with integer 

10 coordinates. A filtered intensity value I(n) corresponding to said point pl(n) is then derived 
according to a step of filtering a set of texels of the input triangles surrounding said input 
transformed point p 1 (n). 

As a first example, the filtering step is based on the use of a bilinear filter adapted to 
implement a bilinear interpolation. Said bilinear interpolation is illustrated by the zoom in 

15 Figure 2 and is based, for example, on the use of the texture values I(tl) to I(t4) of the 4 

texels tl to t4 surrounding the input transformed point pl(n) and on the distance al to a4 of 
said point pi (n) with respect to said 4 texels. The filtered intensity value I(n) is then derived 
as follows: 

I(n) = al J(tl) + a2 J(t2) + a3 J(t3) + a4.I(t4) 
al+a2+a3+a4 

20 This bilinear intetpolation is particularly simple to implement. It will be apparent to the one 
skilled in the art that the bilinear interpolation can be computed according to other equivalent 
principles. For example, the coefficients al to a4 can also correspond to the surface of four 
quadrants delimited by a horizontal and a vertical line passing through the input transformed 
point pi (n). 

25 As another example, the filtering step is based on the use of two mono-dimensional 

finite impulse response FIR filters applied sequentially in a horizontal direction and in a 
vertical direction. The filter taps are determined on the basis of the impulse response IR1 or 
IR2 which is centered on the input transformed point pl(n) according to a given direction, i.e* 
horizontal or vertical, and of the grid GR of texels with integer coordinates, as illustrated in 

30 Figure 3 in the case of a four taps filter. This results in four coefficients hi to h4 and vl to v4 
for each FIR filter. The horizontal FIR filter is applied to the four horizontal segments of the 
set of 16 texels, resulting in 4 intermediate filtered intensity values. Then, the vertical FIR 
filter is applied to the 4 intermediate filtered intensity values, resulting in a final filtered 
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intensity value I(n). It will be apparent to a person skilled in the art that the number of filter 
taps is not limited to 4 and that said number may be increased or decreased. Other filters can 
also be applied, such a polyphase filter for example, depending on the desired visual quality 
and/or the available computing power. 

Once the filtered intensity value I(n) associated with an intermediate point pO(n) of 
the intermediate rectangle triangle TO has been determined, the mapping method in 
accordance with the invention is adapted to determine, for said intermediate point pO(n), an 
output transformed point p2(n) in the output triangle T2 to which the filtered intensity value 
is then associated, using the direct affine transform FT. 

The mapping method in accordance with the invention then comprises a step of 
calculating a contribution to a set of pixels surrounding said output transformed point p2(n) 
on the basis of the filtered intensity value I(n) and of a location of the output point with 

respect to said set of pixels. 

As a first example, the calculating step is based on the distance of the output 
transformed point p2(n) with respect to a set of pixels belonging to a grid of pixels with 
integer coordinates and surrounding said output transformed point For example, if dl to d4 
are the distances between the output transformed point p2(n) and the 4 pixels surrounding 
said point p2(n), as shown in the zoom in Figure 2, the four cortributions jl(n) to j4(n) are 
calculated as follows: 

where D = dl + d2 + d3 + d4 and where i = 1, 2, 3 or 4 

Said calculating step can also include a sub-step of weighting the contribution ji(n) by 
a surface S2 corresponding to the direct affine transform FT of the unitary surface SO 
associated with the intermediate point pO(n). The contribution is then computed as follows: 

j . (n) = S2 .£^I(n) = k i .I(n) (9) 

For a given pixel, the contributions can come from different output transformed 
points. Said contributions are added for said given pixel until the sum of the coefficients ki is 
equal to 1. 

Figure 4 illustrates another embodiment of the mapping method in accordance with 
the invention. The mapping method comprises a step of deterrnining a first set of coefficients 
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which defines the inverse affine transform BT, said inverse transform being capable of 
transforming an intermediate rectangle triangle into an input triangle. Said mapping method 
also comprises a step of determining a second set of affine coefficients corresponding to the 
direct affine transform FT, said direct transform being adapted to transform the intermediate 
5 rectangle triangle into an output triangle. 

Once the two sets of coefficients have been computed, the mapping method in 
accordance with said another embodiment is adapted to determine, for an intermediate point 
pO(n) of the intermediate rectangle triangle TO, an output transformed point p2(n) in the 
output triangle using the direct affine transform FT. Said transform is also adapted to 
1 0 transform a surface S0(n) formed by intermediate points into a surface S2(n). Then a pixel 
pix(n) with integer coordinates in the (x,y) coordinate system belonging to the surface S2(n), 
if any, is determined. The output vector V2(p,q) corresponding to the translation of the output 
transformed point p2(n) towards the translated pixel pix(n) is thus computed. The 
corresponding intermediate vector VO(p\q') in the intermediate space is thus derived as 
15 follows: 

p, T§r - «=m 

A translation of the intermediate point pO(n) towards a translated intermediate point 
p'0(n) according to the intermediate vector VO is then performed. The mapping method in 
accordance with the invention is then adapted to determine, for said translated intermediate 

20 point p'0(n), an input transformed point p' 1 (n) in the input triangle using the inverse affine 
transform BT (see continuous arrow in Figure 4). A filtered intensity value I(n) 
corresponding to said point p'l(n) is then derived from a step of filtering a set of texels of the 
input triangle surrounding said input transformed point p' l(n), as described before. Said 
another embodiment only requires a filtering step compared to the first embodiment and thus 

25 saves computing resources. 

Alternatively, the translated intermediate point p'0(n) is not computed. Instead, the 
mapping method is adapted to determine, for the intermediate point p0(n), an input 
transformed point pl(n) in the input triangle using the inverse affine transform BT (see 
continuous arrow in Figure 4). An input vector Vl(p",q") corresponding to the intermediate 

30 vector V0(p',q') in the intermediate space is thus derived as follows: 
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A translated input transformed point p'l(n) is then determined using the input vector 
Vl(p",q") and a filtered intensity value I(n) corresponding to said point p'l(n) is then derived 
from a step of filtering a set of texels of the input triangle surrounding said point p'l(n), as 
described before. 

The mapping method in accordance with the invention creates an intermediate scan- 
conversion space and is adapted: 

to perform a backward mapping from the intermediate space to the input image so as 
to determine intermediate intensity values corresponding to points of said intermediate space 
on the basis of input intensity values of texels of Hie input image, and 

to perform a forward mapping from the intermediate space to the output image so as 
to determine intensity values corresponding to pixels of said output image on the basis of the 
intermediate intensity values. 

Said mapping method can be refined according to different embodiments. 

According to a first embodiment of the invention, the intermediate rectangle triangle 
is defined by lengths a and b, which are the lengths of the edges aligned with the s and t axis, 
i.e. opposite to the hypotenuse. These two lengths a and b are preferably chosen as a power of 
2 in order to simplify the affine coefficients computing. 

In order to assure no hole in the output image the vectors based on affine coefficients 

( $L *0 f I are also preferably chosen as having a length lower than 1 . For 

example, the two lengths a and b equal to a power of 2 greater than the length of the 
corresponding edges el and e2 in the output triangle. 

In some particular cases, there can be a great distortion between the intermediate 
rectangle triangle and the output triangle. Figure 5 illustrates such a bad case for the 
computation power requested for scan-conversion. The area of the intermediate triangle is 
much larger than the output triangle so that the number of scanned intermediate points is 
unbalanced compared with the actual number of pixels to be computed. That is why, 
according to another embodiment of the invention, the output triangle is cut into two sub- 
triangles. A new vertex is then introduced on the longest edge of the output triangle, as 
shown in Figure 6, for example at the basis of one of the triangle altitudes. In Figure 6, the 
intermediate points (sO,tO), (sl,tl), (s2,t2) and (s3,t3) correspond to the output transformed 
points (xO,yO), (xl,yl), (x2,y2) and (x3,y3), respectively. Then, the scaibconversion of the 
intermediate triangle is performed. For an intermediate point, an input transformed point in 
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the input image is determined. The texels surrounding said input transformed point are 
subsequently filtered, as described before. The position of the output transformed point in the 
output triangle is also determined. The intermediate points are then re-sampled using an 
input-driven filter, as described before, said filter generating the intensity values of pixels 
positioned onto a grid with integer coordinates in the output image. Mapping a rectangle 
triangle to another one assures a well-balanced ratio between intermediate points and pixels. 

Figure 7 shows a block diagram of a rendering device in accordance with the 
invention. Said rendering device is based on a hardware coprocessor realization. This 
coprocessor is assumed to be part of a shared memory system. The dynamic memory access 
DMA unit interfaces the coprocessor with a main memory (not represented). A controller 
CTRL controls the internal process scheduling. An input memory IM contains a local copy of 
the input image. An initialization INIT unit accesses geometric parameters, i.e. the vertices of 
the different triangles, through the DMA unit From said geometric parameters, the INIT unit 
computes the affine coefficients and the edge slopes for the scan-conversion process. 

These geometric parameters are then processed by the rendering REND unit, which is 
in charge of scan-converting the intermediate triangle setting-up during the initialization 
process by the INIT unit. The result of the scan-conversion process is stored in a local output 
memory OM. 

The rendering REND unit comprises a triangle scan TSC unit, which is adapted to 
compute, for each pixel pO of a line of the intermediate triangle, the corresponding location 
pi within the input image and the corresponding location p2 within the output image. The 
coordinates of the input point pi are subsequently used to address the input memory IM to 
get a set of texels with integer coordinates (u,v) to be filtered by a first filter INF unit, 
resulting in a filtered intensity value. Said filtered intensity value is provided together with 
the coordinates of the output point p2 to the inputs of a second filter OUTF unit, which is 
input driven. Said second filter unit is adapted to compute contributions to pixels of the 
output image having integer coordinates (x,y) and surrounding the output point p2. The 
output memory OM stores accumulated contributions. Once all the triangles have been 
rendered, the output memory is written back into a system memory, a video controller being 
adapted to access the output image in said system memory to display it. 

The mapping method in accordance with the invention can be implemented by means 
of items of hardware or software, or both. Said hardware or software items can be 
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implemented in several manners, such as by means of wired electronic circuits or by means 
of an integrated circuit that is suitable programmed, respectively. The integrated circuit can 
be contained in a portable apparatus. The integrated circuit comprises a set of instructions. 
Thus, said set of instructions contained, for example, in a memory of the portable apparatus 
5 may cause the integrated circuit to carry out the different steps of the mapping method. The 
set of instructions may be loaded into the memory by reading a data carrier such as, for 
example, a disk. A service provider can also make the set of instructions available via a 
communication network such as, for example, the Internet. 

10 Any reference sign in the following claims should not be construed as limiting the 

claim. It will be obvious that the use of Hie verb "to comprise" and its conjugations do not 
exclude the presence of any other steps or elements besides those defined in any claim. The 
word "a 95 or "an" preceding an element or step does not exclude the presence of a plurality of 
such elements or steps. 



15 
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CLAIMS 

1 . A method of mapping an input image split into input triangles including texels onto an 
output image also split into corresponding output triangles including pixels, said method 
comprising tire steps of: 

determining an inverse affine transform (BT) for transforming an intermediate 
rectangle triangle (TO) into an input triangle (Tl); 

determining a direct affine transform (FT) for transforming the intermediate rectangle 
triangle (TO) into an output triangle (T2); 

applying the inverse affine transfonn to intermediate points of the intermediate 
rectangle triangle (TO) so as to determine intermediate intensity values corresponding to said 
intermediate points on the basis of input intensity values of texels; and 

applying the direct affme transform to the intermediate points so as to determine 
output intensity values of pixels on the basis of the intermediate intensity values. 

2. A method as claimed in claim 1 , wherein the step of applying the inverse affine 
transform is adapted to transform an intermediate point (pO(n)) into an input transformed 
point (pl(n)) in the input triangle, and to determine, for said intermediate point, an 
intermediate intensity value based on a filtering operation of texels surrounding the input 
transformed point. 

3 . A method as claimed in claim 2, wherein the filtering operation comprises a bilinear 
interpolation using four texels surrounding the input transformed point. 

4. A method as claimed in claim 2, wherein the filtering operation comprises applying 
sequentially a first mono-dimensional finite impulse response filter in a horizontal direction 
and a second mono-dimensional finite impulse response filter in a vertical direction. 

5. A method as claimed in claim 1, wherein the step of applying the direct affine 
transform is adapted to transform an intermediate point (pO(n)) into an output transformed 
point (p2(n)) in the output triangle, and to determine, for said intermediate point, a 
contribution to output intensity values of pixels surrounding said output transformed point o: 
the basis of the intermediate intensity value. 
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6. A method as claimed in claim 1, further comprising a step of deterrnining lengths 
(a,b) of the intermediate rectangle triangle opposite to the hypotenuse which are equal to a 
power of 2 greater than the length of corresponding edges (el,e2) in the output triangle. 

5 7. A method as claimed in claim 1 , further comprising a step of dividing the output 
triangle into two sub-triangles before the step of applying the direct affine transform. 

8. A method as claimed in claim 1 , wherein: 

the step of applying the direct affine transform is adapted to determine an output point 
1 0 (p2(n)) and a corresponding output surface (S2(n)) in the output triangle (T2) from an 
intermediate point (pO(n)) and a corresponding intermediate unitary surface (S0(n)), to 
determine a pixel (pix(n)) with integer coordinates belonging to the output surface, and to 
determine an output vector (V2) defined by the output point and the pixel with integer 
coordinates; and 

15 - the step of applying the inverse affine transform is adapted to determine an input 
transformed point (p'l(n)) in the input triangle (Tl) from the intermediate point and the 
output vector, and to filter the input intensity values of texels surrounding said input 
transformed point so as to derive an output intensity value of the pixel with integer 
coordinates. 

20 

9. A device for rendering an output image split into corresponding output triangles 
including pixels on the basis of textured data of an input image split into input triangles 
including texels, said device comprising: 

means (INIT) for determining an inverse affine transform (BT) for transforming an 

25 intermediate rectangle triangle into an input triangle, and for determining a direct affine 
transform (FT) for transforming an intermediate rectangle triangle into an output triangle; 

means (TSC,IM,INF) for applying the inverse affine transform to intermediate points 
of the intermediate rectangle triangle (TO) so as to determine intermediate intensity values 
corresponding to said predetermined points on the basis of input intensity values of texels; 

30 and 

means (TSC,OM,OUTF) for applying the direct affine transform to the intermediate 
points so as to determine output intensity values of pixels on the basis of the intermediate 
intensity values. 
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10. A portable apparatus comprising a device as claimed in claim 9, 

1 L A computer program product comprising program instructions for implementing, 
when said program is executed by a processor, a method as claimed in claim 1 . 
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Method of rendering graphical objects 
ABSTRACT 

A method of mapping an input image split into input triangles including texels onto 
5 an oulput image also split into corresponding output triangles including pixels. Said method 
comprising the steps of: 

- * determining an inverse affine transform (BT) for transforming an intermediate 
rectangle triangle (TO) into an input triangle (Tl); 

determining a direct affine transform (FT) for transforming the intermediate rectangle 

10 triangle (TO) into an output triangle (T2); 

- applying the inverse affine transform to intermediate points of the intermediate 
rectangle triangle (TO) so as to determine intermediate intensity values corresponding to said 
intermediate points on the basis of input intensity values of texels; and 

applying the direct affine transform to the intermediate points so as to determine 
15 output intensity values of pixels on the basis of the intermediate intensity values. 



Reference: Figure 2 
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